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Outline of Presentation 


• History & Background Info on SAT Writing 

• Recent National Validity Information on SAT Writing 

• Incorporating SAT Writing into Admissions - Paul Johnson, 
Rutgers University 

• Incorporating SAT Writing into Placement - Kathleen 
O'Connor, Lasell College 

• Conducting Local Admission and/or Placement Validity 
Studies 

• Questions 
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What is the SAT writing section? 


• Implemented in March 2005. 

• Lasts one hour, and is scored on a scale of 200 to 
800. 

• Multiple choice questions test a student's ability to 
improve sentences and identify sentence errors 

• This section accounts for 70% of the writing score 

• Students have 35 minutes to answer 49 multiple choice questions 

• The essay tests a student's ability to articulate a 
coherent argument, supporting a point of view on an 
issue specified on the test 

• The essay accounts for 30% of the writing score 

• Students have 25 minutes to write the essay 
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And how is the test performing?... 
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National SAT Validity Research 


• Examines the relationship of SAT Writing with 
important college outcomes, including: 

• SAT Writing and FYGPA 

• SAT Writing and FY English grades 

• SAT Writing and Retention 

• SAT Writing and cumulative GPA 


Studies can be found at: 
www.colleaeboard.com/research 
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Correlation 


• Essentially, a correlation coefficient is a number 
between -1 and 1 which measures the degree to 
which two variables are linearly related. 


Strength (absolute magnitude) and direction (negative/positive) 
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Rule of Thumb for Interpreting Correlation Coefficients 


• A general rule of thumb for interpreting 
correlation coefficients is offered by Cohen 
( 1988 ): 

• small correlation has an absolute value of 
approximately 0.1 

• medium correlation has an absolute value of 
approximately 0.3 

• strong correlation has an absolute value of 
approximately 0.5 or higher 
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SAT Validity Study results - snapshot 



Restrict sample to students who 
completed the new SAT, submitted 
self reported HSGPA, and had a valid 
FYGPA (N=1 73,963) 


• Admission Validity Study 


• Schools provided first year performance data 
for Fall 2008 cohort through the Admitted 
Class Evaluation Service™ (ACES ™) portal 


SAMPLE (2008 entering cohort) 

129 colleges participating in Validity 
Study (N = 246,652) 
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Admission Validity Results (1 of 2) 


• SAT Writing has the highest correlation with 
FYGPA among the three individual SAT 
sections ( Adj . r= 0.52). 

• SAT CR (Adj. r= 0.48); SAT M (Adj. r= 0.48) 

• As expected, the best combination of 
predictors of FYGPA is HSGPA and SAT 
scores (Adj. r= 0.63), reinforcing the 
recommendation that colleges use both 
HSGPA and SAT scores to make the best 
predictions of student success. 
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What does a validity coefficient of .53 mean? 
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Admission Validity Results (2 of 2) 


• The increment in validity attributable to the 
Writing section over and above the CR and M 
sections is 0.02. When HSGPA is also 
considered, the increment in validity attributable 
to the Writing section is 0.01 . 


Let's look at this graphically.... 
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Percent of Students 


Contribution of SAT Writing to Predicting Success for Students 
Lower Achievers: HSGPA < 3.3 and SAT CR+M < 1000 
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Contribution of SAT Writing to Predicting Success for Students 
Average Achievers: HSGPA between 3, 3-3.7 and SAT CR+M 

between 1010 and 1200 



■ % FYGPA 3.0 or 
above 

■ % FYGPA 3.5 or 
above 



SAT Writing Score 


Percent of Students 


Contribution of SAT Writing to Predicting Success for 

Students 

High Achievers: HSGPA > 3.7 and SAT CR+M > 1200 
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■ % FYGPA 3.5 or 
above 
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English Course Grade 


Relationship between SAT Critical Reading and 
Writing and 1st Year English Course Grades 


The Relationship between SAT-CR & W Scores and 
Grades Earned in 1st Year English Courses 



SATScore Band 
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■ SAT-CR ■ SAT-W 
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% of Students with B or Higher 


Relationship between SAT CR & W and Earning 
a B or Higher in 1 st Year English Course 


Percentage of Students Earning a B or Higher in 1st Year English 

Courses by SAT-CR & W Scores 



SATScore Band 
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■ SAT-CR ■ SAT-W 
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SAT Writing and Retention 


Contribution of SAT Writing in the Prediction of 
Retention to Second Year Controlling for HSGPA and 



HSGPA and SAT-M & CR student groups 


□ SAT-W of 500 and below □ SAT-W between 51 0 - 600 


□ SAT-W of 610 and higher 
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SAT Writing and Cum GPA 


•The SAT Writing section remains quite predictive 
of cumulative GPA at the end of the third-year of 
college. ( adj. . r = 0.56) 


Correlation 


Predictor(s) 

1. HSGPA 

2. SAT-CR 

3. SAT-M 

4. SAT-W 

5. SAT-M, SAT-CR 

6. HSGPA, SAT-M, SAT-CR 

7. SAT-CR, SAT-M, SAT-W 

8. HSGPA, SAT-CR, SAT-M, 


3 rd Yr Cum GPA 

0.57 

0.52 

0.50 

0.56 

0.55 

0.65 

0.58 

-W 0.66 


Note. N = number of students = 63,736. Pooled within-institution, restriction of range corrected correlations are 
presented; raw correlations are shown in parentheses. 
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Next... 


Incorporating the SAT into Admissions at 

Rutgers University 
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Next... 


Incorporating the SAT into English 
placement decisions at Lasell College 
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Conducting a Local Admission and/or 
Placement Validity Study on SAT Writing 


Admitted Class Evaluation Service (ACES) 

• The Admitted Class Evaluation Service 
(ACES) is a free online service that predicts 
how admitted students will perform at a 
college or university generally (admission 
validity) and how successful students will 
be in specific classes (placement validity). 

http://professionals.colleqeboard.com/hi 

qher-ed/validity/aces 
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ACES Admission Validity Studies 


• The primary purpose of an admission validity study is to 
validate measures used in admission decisions. 

• Can determine how well admission criteria work alone and 
in combination with other predictors, and the most effective 
weighting for the predictors. 

• Success (the criterion) may be measured by college GPA 

• Relevant predictors may be 

• SAT scores - Critical Reading, Math, or Writing 

• High school GPA, or Class Rank 

• Interview scores, and 

• Other information 



Overview of ACES Process 


1. 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 


institutional contact/submitter will: 

Click link on ACES web site for a new ACES study request: 
https://cbweb1s.colleqeboard.org/aces/html/newrvs.htnnl 

Enter contact info (name, email, position, institution, etc.) 

Design study (choose predictors, subgroups, etc.) 

Receive automatically e-mailed user account, password, and request 
number from ACES 

Login to submit data at this site: 

https ://cbweb1 s.colleqeboard.orq/aces/html/submitl .html 
Record all variable locations, indicate value labels, etc. 

Upload data file(s) 

ACES reports are returned to institutions 25 - 35 business days after the 
receipt of data. 



ACES Admission Validity Report 


ACES 


toman *«•» 


Admission Validity 
Report for Sample 
One University 

Bat* it- tu< report an not ripciHcmiv* oc any iMtltatm- All s*t» art 
Bypotmcitai M w*r« for tn# *014 purport or erwitsg era* **^pi* report - 

Entering Class of Fall 2007 

Your College Board Validity Report is designed to assist your 
wiiirtutionin validating yow admission decisions. This report 
provides a nontechnical discussion of important findings 


ADMITTED CLASS EVALUATION SERVICE 1 ™ 

WWW.CO LLEGEBO ARD COM 


f CoIiegeBoard 

connect to college success™ 
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Section 1: Evaluating individual admission measures 


Section 1: Evaluating Individual admission measures 

This section summarizes the predictive strength of the individual admission measures in your study, first for the measures available 
for most of your students, and then for measures available for smallergroups of students. The second analysis may include results 
for predictors, such as SAT Subject Tests, that you did not explicitly choose to study but were present in your students' records. You 
may wish to consider the use of this additional information for future admission decisions. 

See Section 2 for combinations of the individual measures, which are likely to provide more reliable and fairer information on your 
applicants. 

The tables below display the absolute value of correlations between each admission measure and First-Year GPA, the criterion you 
chose for this study. 


Individual admission measures in your study 


Strong Ftedktors 

N 

Predictive Strength icorTolationj 

HS Rank 

978 

Has 

SAT Critical Heading 

978 

■ctiz 

SAT Writing 

978 

Hd.42 

Moderate Predictor:- 

SAT Math 

978 


a.E9 

#AP Exam:- 

7EE 


a.29 

SATStij: High-NcnLang 

782 

”” 1 0.38 

* EAT Eubj Tests 

782 

EaWd 3 $ 

Weak Predictors 

£ Honcrs or AP courses | 7E2| Jo.22 


Other admission measi res available 


Strong Predictors 

N 

Predictive Strength Icorrelationj 

SATStirj: Math Level 1 

24 i 

■ 0.45 

SATStirj:U.E. History 

115 

Hdji 

Moderate Predictors. 




SATStij: Literature 

“1 

1 ° J1 


Weak Predictors 

None available 


Notes; 

* All individual measures have moderate to strong correlations with First-Year GPA except for the # Honors or AP courses measure. 
The measures showing moderate to strong correlations with First- Year GPA are good candidates for inclusion in the predicted 
First-Year GPA calculations in Section Z. 
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This section summarizes the 
predictive strength of the 
individual admission measures 
in the study. 

The second analysis may 
include results for predictors, 
such as SAT Subject Tests, 
that institutions did not 
explicitly choose to study but 
were present in their 
students' records. 
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Section 2: Evaluating combined admission measures 


Section 2: Evaluating combined admission measures 

This section combines the admission measures that were evaluated individually in Section 1 of this report to find the best prediction 
of success. Combinations that are available for most of your students are presented first,, followed by combinations that are 
available forsmaller subgroups. 

Because combinations of predictors tend to be more reliable and allow students to shojw different strengths, it is important to 
consider all of the information available fora given student in making an admission decision. Appendix A presents the equations 
needed to combine the admission measures into a single predicted First-Year GPA. Severn I equations are given so that you can use 
as much of the information provided to you by each student as possible. This section of your report gives you the information you 
need to choose the best combination of predictors for each student. 

The tables below display the multiple correlations between combinations of admission measures and the measure of success you 
chose for this study. The bars at the right of each table represent this predictive strength (multiple correlation] foi each combination. 

The first table below presents SAT combinations. The first line erf that table shows the multiple correlation for the predicted First- 
Year GPA using only SAT scores. 


SAT combinatiois 


SAT 

HS Hut 

Add. 

FVirinor: 

N 

Prndktr,; [length |m id jit [nrablion' 

Critical 

nudity 

r-t.fi 

Mil rig 

H 

31 

34 



979 

DjH 


17 

3D 

* 


979 


17 

19 

3D 

41 

1 

371 



SAT & SAT Subject Tesl combination 


SAT 

SAT Slit pH 
Tesl: 

HSHsrt 

Add. 

ftrikbis 

N 

PtsdkdvB Si;«igdi |nidti[lt twraliliav 

OpcisI 

taring 

Hah 

Writing 

Hqh- 

KorLmg 

a 

Z7 

57 | 

r~ 



m 

■ ['.Ed 

17 

19 

a 

15 

E 


m\ 

' (USD 

19 

a 

54 

11 

a 

1 

raz 

Booi 


Notes: 

* The multiple correlation calculated by using SAT Math,. SAT Critical Reading, and SAT Writing was 0.44, which represents a 
strong correlation. I he numbers in the boxes to the left of the bars show the relative contribution of each piedictor I in percentage 
terms] for each prediction equation. SAT Critical Reading contributes 3G percent, SAT Math contributes 31 percent, and SAT 
Writing contributes 34 percent when using the SAT in predicting First- Year GPA. 

* The second line of the SAT combinations table adds FIS Rank to the SAT information. Of the SAT and FIS Rank. FIS Flank makes 
the greatest contribution toward predicting First-Year GPA. After adding HS Rank, the multiple correlation increased from 0.44 to 
0.57, 
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This section combines the admission 
measures that were evaluated 
individually in Section 1 of the report 
to find the best prediction of 
success. 

These tables display the multiple 
correlations between combinations 
of admission measures and the 
criterion. The bars at the right of each 
table represent this predictive 
strength (multiple correlation) for 
each combination. 
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ACES Placement Validity Studies 


• ACES Placement Validity Studies give you the information 
you need to confirm or improve your current course 
placement policies. 

• Predictive placement validity studies evaluate student 
scores on the SAT, for example, to predict performance in 
a particular course. 

• The study design is used to determine the score level on 
the test at which students should be placed into a course. 
This cutoff score can then be applied to future students 
who have taken the test in order to place into the course. 
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ACES Placement Validity Study 


• The predictive validity study provides two 
probability tables for cut scores, among other 
information: 

• For a course grade of B or higher 

• For a course grade of C or higher 


CollegeBoard 

' inspiring mindsT 


Example of ACES Placement Chart 


Cut Scores Associated with Predicted Probability of Success Criterion: 
Final Course Grade of C or Higher in Engl 00 Using SAT Scores 


Cut Scnns Associated with Predicted Probability ol Six cots 
Criterion: Hml Court* Cud* ol C or Highor ii I ng 100 
Using SAT Scons 


PrfttMbtav 
of Success 

SAT Criic*l Ruling 0 ink 

S4J WriiagOtfy 

Corptut* 

0.96 



tm 

0.90 

791 


220 

0.86 

692 

750 

173 

m 

621 

649 

UQ 

0.76 

556 

570 

110 

0.70 

492 

512 

006 

0.66 

443 

476 

062 

0.60 

390 

416 

0.41 

0.56 

345 

370 

0.20 

050 

300 

327 

000 

o.« 

256 

260 

-020 

0.40 

211 

227 

4.41 

0.36 



462 

0.30 



405 

0.25 



110 

020 



•U0 

0.15 



473 

Q.W 



220 

0.06 



•294 


Th* fallowing tnodtltl can b«u*d to celculiie tree twit stop*, 'or. rwninlheiabfaiber* 


Mod* Nuntw 1 ■ J 23677 *(0 005(61 > SAT Cntictl ►. xJ.ng., 000621 ■ ■ SAT.Vmng 
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Chart with Correlations and % Correctly Placed 


Individual predictors examined (SAT CR alone, SAT W alone) 
Composite predictor examined (SAT CR and W together) 


Logtstic Bi serial Correlations' of Predictors will Success on tire Criterion 



Criterion: Final Course tirade ol Cor Higher in EnglQO 





Using SAT Scores 





Study Sample 

Complete Data Semple 

FTediclor Variable) -si 

N 

Logistic Bisarial 

% Correctly 

N 

Logistic Biserial 

% Correctly 



Correlation’' 

Flaced 


Correlation *■ 

Flaced 

Individual Predictors 

SAT Critical Reading 

tee 

0.1 S 

00 

402 

0.1 B 

00 

SAT Writing 

492 

020 

70 

402 

020 

70 

Composite Predictors 

Model Number 1 

492 

0.37 

6 B 

402 

0.37 

63 


Model Number 1 includes SAT Critical Raiding end CAT Wrh r.g 


"The logistic biserial correlation is i measure of the slrength of associaliun. h is related to a biserial [DiTelatian but has bean muddle-: In Le consistent with logistic regression and 
adapled In single end miJti|)le predictors 
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ACES Validity Handbook 

(http://professionals.collegeboard.com/higher-ed/validity/aces/handbook) 


• It is designed to serve as a 
general reference for validity 
and includes information about 
validity beyond what is 
specifically applicable to ACES. 


• It includes specific information 
about the types of validity 
studies and their design that are 
available through the ACES 
system and helps interpret 
ACES study results. 


Validity 

ACES’* 



Admission Vatdty Study 
Placement VabdCy Study 
Validity Handbook 
Test VaWty 
Vabdfly Evidence 
Glossary 

Which Study ts Right for You 

About Admission VaWCy 
Studies 


About Placement Vaidty Studes 

Predictive Placement Vahdfy 
Studies 

Concurrent Placement VafcdCy 
Studies 

Existing Placement Program 
Evaluation 

Avoidng Potential Problems 
FAQ 
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The End 


• Questions? 

• Thank you for joining us today! 


Paul Johnson 

Rutgers University, NJ 
Johnson@uaadm.rutaers.edu 

Kathleen O'Connor 

Lasell College, MA 
KOConnor@Lasell.edu 

Emily Shaw 

The College Board, NY 
eshaw@colleaeboard.org 

Researchers are encouraged to freely express their opinion. The results here do not necessarily 

represent the opinion or views of The College Board. ^ 
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